Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering

Retrieval Augment Generation (RAG) is a recent advancement in Open-Domain Question Answering (ODQA). RAG has only been trained and explored with a Wikipedia-based external knowledge base and is not optimized for use in other specialized domains such as healthcare and news. In this paper, we evaluate the impact of joint training of the retriever and generator components of RAG for the task of domain adaptation in ODQA. We propose RAG-end2end, an extension to RAG that can adapt to a domain-specific knowledge base by updating all components of the external knowledge base during training. In addition, we introduce an auxiliary training signal to inject more domain-specific knowledge. This auxiliary signal forces RAG-end2end to reconstruct a given sentence by accessing the relevant information from the external knowledge base. Our novel contribution is that, unlike RAG, RAG-end2end does joint training of the retriever and generator for the end QA task and domain adaptation. We evaluate our approach with datasets from three domains: COVID-19, News, and Conversations, and achieve sig-nificant performance improvements compared to the original RAG model. Our work has been open-sourced through the HuggingFace Transformers library, attesting to our work's credibility and technical consistency. © 2023 Association for Computational Linguistics. Distributed under a CC-BY 4.0 license.

Novel insight from the first lung transplant of a COVID-19 patient.

BACKGROUND: To reveal detailed histopathological changes, virus distributions, immunologic properties and multi-omic features caused by SARS-CoV-2 in the explanted lungs from the world's first successful lung transplantation of a COVID-19 patient. MATERIALS AND METHODS: A total of 36 samples were collected from the lungs. Histopathological features and virus distribution were observed by optical microscope and transmission electron microscope (TEM). Immune cells were detected by flow cytometry and immunohistochemistry. Transcriptome and proteome approaches were used to investigate main biological processes involved in COVID-19-associated pulmonary fibrosis. RESULTS: The histopathological changes of the lung tissues were characterized by extensive pulmonary interstitial fibrosis and haemorrhage. Viral particles were observed in the cytoplasm of macrophages. CD3+ CD4- T cells, neutrophils, NK cells, γ/δ T cells and monocytes, but not B cells, were abundant in the lungs. Higher levels of proinflammatory cytokines iNOS, IL-1ß and IL-6 were in the area of mild fibrosis. Multi-omics analyses revealed a total of 126 out of 20,356 significant different transcription and 114 out of 8,493 protein expression in lung samples with mild and severe fibrosis, most of which were related to fibrosis and inflammation. CONCLUSIONS: Our results provide novel insight that the significant neutrophil/ CD3+ CD4- T cell/ macrophage activation leads to cytokine storm and severe fibrosis in the lungs of COVID-19 patient and may contribute to a better understanding of COVID-19 pathogenesis.